1. Field of the Invention
This invention relates generally to optical subsystems that are used for imaging and, more particularly, to subsystems that provide information about the image quality produced by the subsystem.
2. Description of the Related Art
Electro-optic imaging systems typically include an optical subsystem (e.g., a lens assembly), an electronic detector subsystem (e.g., CCD detector array) and a digital image processing subsystem (e.g., typically implemented in dedicated chips or software). For many commercial imaging systems, the market has developed in a manner where different components are supplied by different vendors. For example, it is not unusual for one set of vendors to provide different types of optical subsystems while a different set of vendors provides the electrical back-end subsystem (i.e., the detector subsystem plus the digital image processing subsystem, if any).
One example is the market for digital cameras. Digital cameras may be assembled from components supplied by different vendors, specifically camera bodies with interchangeable lenses. Likewise, in digital still cameras, such as those found in cell phones, camera integrators commonly buy off-the-shelf lens subsystems on the open market for integration with their back-ends.
This separation of optics vendors from electrical back-end vendors is feasible in part because, under traditional design methods, the optical subsystem is designed independently of the electrical back-end. Typically, the optical subsystem is designed with the goal of forming a high quality optical image of the object. In many cases, the goal is to design a diffraction-limited optical subsystem. As a result, for purposes of designing the electrical back-end, the optical subsystem often can be adequately described solely by parameters that describe the overall imaging scenario: F/#, focal length and conjugate and field size. Standardized mechanical mounts farther facilitate the interchangeability of different optical subsystems and the separate design of optical subsystem and electrical back-end.
One advantage of this separated approach is that companies can specialize in either optical subsystems or electrical back-ends. One vendor might build up expertise in optical subsystems, thus allowing it to produce better quality products at a lower price. Furthermore, if the product has a standardized mechanical mount, it can be used with a number of different electrical back-ends produced by different vendors. This increases the overall volume for the product, further reducing the cost.
However, one drawback of this conventional approach is that synergies between the optical subsystem and the electrical back-end may be lost. For example, a number of vendors may produce competing products with the same F/#, focal length, etc. However, each vendor's product will likely be a different design with different image qualities (e.g., slightly different blurring, distortion, etc.). Furthermore, these image qualities may differ between individual subsystems from the same vendor due to manufacturing variations. However, if the optical subsystem is described simply by its F/#, focal length, etc., the information about image quality will be lost. In some cases, the image processing capability of the back-end may be able to compensate for image degradation caused by the optics but, without information about the image quality, the image processing subsystem will not know what to do.
One example is the market for SLR camera lenses, where multiple lens vendors supply lenses designed to work with a variety of camera bodies. These lenses are interchangeable by the end user. The current image processing subsystems in the camera body rarely, if ever, utilize any information about the quality of the image produced by the lens. This is partly because that information is simply not made available to the image processing subsystem. Instead, the camera body is designed for a “generic” (usually, high quality) lens and uses the corresponding “generic” image processing. The system relies on high performance (and expensive) optics to achieve acceptable performance. There generally is no ability for the image processing subsystem to adapt as a function of the lens attached to the camera body. This is partly because, in order to facilitate interchangeability, lenses are designed to the lowest common denominator. For example, in many cases, this lowest common denominator is diffraction-limited so that the electrical back-end can simply be designed to match diffraction-limited optics.
However, this can be a significant drawback since recent developments demonstrate there can be synergies between the optical subsystem and the electrical back-end. In some cases, a simpler, less expensive, non-diffraction limited lens can achieve the same overall system performance as a diffraction limited lens because the image processing compensates for shortfalls in the lens. The following co-pending patent applications generally concern the end-to-end design of electro-optic imaging systems where the optical subsystem and digital image processing subsystem are designed together to take advantage of synergies between the two: U.S. application Ser. No. 11/155,870, “End-To-End Design Of Electro-Optic Imaging Systems,” filed Jun. 17, 2005; Ser. No. 11/245,563, “Joint Optics And Image Processing Adjustment Of Electro-Optic Imaging Systems,” filed Oct. 7, 2005; Ser. No. 11/332,640, “End-To-End Design Of Electro-Optic Imaging Systems With Constrained Digital Filters,” filed Jan. 13, 2006; Ser. No. 11/433,041, “End-To-End Design Of Superresolution Electro-Optic Imaging Systems,” filed May 12, 2006 and Ser. No. 11/433,780, “End-To-End Design Of Electro-Optic Imaging Systems With Adjustable Optical Cutoff Frequency,” filed May 12, 2006. The foregoing are incorporated by reference herein. In these examples, the capabilities of the digital image processing subsystem are leveraged to generate lens systems which are potentially cheaper and/or offer new imaging capabilities. The image processing and optics in these systems are carefully balanced to interact in a beneficial manner.
More generally, one advantage of digital image processing subsystems compared to optical subsystems is that image processing parameters can usually be changed at a relatively low cost. Based on the examples referenced above, there appear to be a number of situations where this characteristic may be exploited in order to increase overall image system quality using tunable digital image processing subsystems that respond to changes in the optical subsystem.
This ability of the image processing subsystem to change depending on the attached lens system will become even more important as the use of non-traditional optical subsystems proliferates. For example, CDM Optics of Boulder, Colo. has developed a specially designed phase plate that is placed at the aperture of the optical subsystem in order to encode the incoming wavefront in a particular way. Digital image processing is used later to reverse the encoding introduced by the phase plate and retrieve certain image content. In another example, researchers in Osaka, Japan built a multi-aperture imaging system which uses a two-dimensional array of optical lenses to generate an array of low quality optical images at the detector plane. The collection of low quality images are combined using digital image processing to create a high resolution image. In this way, a high resolution image can be obtained using a thin optical system. The vendors for these non-traditional optical subsystems may build lens systems to realize the many benefits of interaction between the optics and the image processing, while still desiring to preserve the ability to interoperate with a broad range of electrical back-end subsystems.
Another example of the usefulness of adaptable image processing subsystems is when the lens system may have variable optical properties as a result of user interaction or camera control (e.g. zoom systems, varifocal systems, etc) or other environmental factors. For example, when the user adjusts the lens system, it may be beneficial for the image processing to change correspondingly.
However, in all of these situations, in order for the image processing to make appropriate adjustments, the electrical back-end requires information about the image quality produced by the optical subsystem. Current systems that simply provide F/#, focal length, etc. simply do not provide this information.